1. Field of the Invention
The present invention relates to an apparatus, method and computer program product for transcoding a coded moving picture sequence, and more particularly, to an apparatus, method and computer program product for converting a first coded moving picture sequence signal in the form of bit streams transmitted at a first bit rate into a second coded moving picture sequence signal in the form of bit streams transmitted at a second bit rate lower than the first bit rate by adapting a quantization parameter to perform the effective real timely rate control over the second coded moving picture sequence signal with low delay.
2. Description of the Related Art
There has so far been proposed a wide variety of systems for compressing and encoding a moving picture having a considerable large data to produce a coded moving picture sequence signal. The international standard, ISO-IEC 13818-2, was created for a system operable to encode a digital video signal associated with a digital audio signal and commonly called xe2x80x9cMoving Picture Expert Group Phase 2xe2x80x9d, i.e., xe2x80x9cMPEG-2xe2x80x9d. In such encoding system, the coded moving picture sequence signal is outputted in the form of bit streams. In particular, the bit streams conformable to above MPEG-2 standard will be referred to as xe2x80x9cMPEG-2 bit streamsxe2x80x9d hereinlater. Recently, the system of this type becomes more utilizable for various technical fields, such as a communicating system, a television broadcasting system, and so on.
The above MPEG-2 bit streams have a hierarchical structure consisting of: in turn, a top, sequence layer, a GROUP OF PICTURES layer, a picture layer; a slice layer; a macroblock layer; and a low, block layer.
According to the MPEG-2 standard, a typical encoder is operable to employ the following method of compressing and encoding a moving picture. The method comprises the steps of:
(a) inputting the moving picture sequence consisting of a series of pictures;
(b) temporally storing the pictures as frames in memories, respectively;
(c) computing a difference between one frame and another frame to eliminate redundancy in a time axis direction; and
(d) orthogonal transforming, e.g., discrete cosine transforming (DCT), a plurality of picture elements within each of the frames to eliminate redundancy in a spatial axis direction.
The encoder thus constructed can compress and encode the moving picture to generate and output the coded moving picture sequence signal in the form of the MPEG-2 bit streams through a transmitting path at a predetermined bit rate. The coded signal is then transmitted from the encoder to a decoder which is adapted to decode the coded signal to reproduce the moving picture. The typical decoder is operated to employ so-called bi-directionally predicting method which comprises the steps of:
(a) storing one reproduced picture, generally referred to as xe2x80x9cintra-picturexe2x80x9d, i.e., xe2x80x9cI-picturexe2x80x9d, in a first frame memory;
(b) estimating another picture generally referred to as xe2x80x9cpredictive-picturexe2x80x9d, i.e., xe2x80x9cP-picturexe2x80x9d, followed by the I-picture, on the basis of the information on a difference between I-picture and P-picture;
(c) storing the estimated picture in a second frame memory; and
(d) estimating further another picture interposed between the I-picture and P-picture, generally referred to as xe2x80x9cbi-directionally predictive-picturexe2x80x9d, i.e., xe2x80x9cB-picturexe2x80x9d.
Here, the I-picture is encoded independently of the other types of pictures, so that the I-picture can be reproduced as a static image only by itself. The P-picture can be predicted on the basis of the I-picture or P-picture located on a position prior to this P-picture to be encoded.
In the above encoder, the volume of information on the coded moving picture sequence signal is, however, variable, in particularly, remarkably increased in accordance with a change of scene. The decoder is generally provided with an input buffer for receiving the coded moving picture sequence signal from the encoder. The input buffer of the decoder, however, has a limited storage capacity. Therefore, when a large number of bits of the coded moving picture sequence signal is transmitted from the encoder to the decoder, the input buffer overflows with the large number of bits of the coded moving picture sequence signal thereby making the decoder hard to process the coded moving picture sequence signal. In order to transmit such coded moving picture sequence signal having a variable volume of information on the transmitting path at a predetermined invariable bit rate and to enable any decoders to receive the coded moving picture sequence signal, the encoder comprises: an output buffer for temporally storing the coded moving picture sequence signal before transmitting the coded moving picture sequence signal on the transmitting path; and a rate controller for controlling the volume of information on the coded moving picture sequence signal stored in the output buffer so as to keep the volume of information on the coded moving picture sequence signal from exceeding the volume of the buffer and then to control the bit rate of the coded moving picture sequence signal.
A typical rate controlling method in MPEG-2 standard is described in xe2x80x9cISO-IEC/JTC1/SC29/WG11/N0400 Test Model 5xe2x80x9d, April, 1993, hereinlater referred to as xe2x80x9cTM-5xe2x80x9d. According to the TM-5, the rate controlling method comprises the steps of:
(I) allocating a target bit by estimating the number of bits available to code the next picture before coding the current picture;
(II) computing the reference value of the quantization parameter used for the quantization of each of macroblocks in the current pictures by means of a xe2x80x9cvirtual bufferxe2x80x9d to perform the rate control; and
(III) modulating the reference quantization parameter in accordance with the spatial activity in the macroblock.
Furthermore, there are many types of decoders, for instance, one is designed to decode the coded signal having a specific format different from that of MPEG-2 bit streams, and another is connectable to a transmitting path having a different bit rate. These types of decoders are therefore required to provide with an apparatus, so-called transcoder, for converting a bit rate of the MPEG-2 bit streams into another appropriate coded moving picture sequence signal. The coded moving picture sequence signal can be transmitted to any types of decoders from the encoder via an appropriate transcoder.
Referring to FIG. 13 of the drawings, there is shown one typical type of the conventional transcoders as a first conventional transcoder 50. The conventional transcoder 50 has an input terminal a1 electrically connected to a first transmitting path, not shown, and an output terminal a2 electrically connected to a second transmitting path, not shown. The conventional transcoder 50 is designed to input first bit streams b1 at a predetermined input bit rate through the input terminal a1, to convert the first bit streams b1 into second bit streams b2 to be output at a predetermined output bit rate, i.e., a target bit rate, lower than the input bit rate of the inputted first bit streams b1, and to then output the second bit streams b2 through the output terminal a2. The conventional transcoder 50 comprises a variable length decoder 51, referred to as xe2x80x9cVLDxe2x80x9d in the drawings, a de-quantizer 53, referred to as xe2x80x9cIQxe2x80x9d in the drawings, a quantizer 55, referred to as xe2x80x9cQxe2x80x9d in the drawings, a variable length encoder 57, referred to as xe2x80x9cVLCxe2x80x9d in the drawings, and a rate controller 59.
The variable length decoder 51 is electrically connected to the input terminal a1 and designed to decode a coded moving picture sequence signal within the first bit streams b1 inputted through the input terminal a1 to reconstruct an original picture data for each of pictures including a matrix of original quantization coefficients, referred to as xe2x80x9clevelxe2x80x9d, for each of macroblocks within each of the pictures and an original quantization parameter, hereinlater referred to as xe2x80x9cfirst quantization parameter Q1xe2x80x9d.
The de-quantizer 53 is electrically connected to the variable length decoder 51 and designed to input the matrix of original quantization coefficients level from the variable length decoder 51 and the first quantization parameter Q1. The de-quantizer 53 is further designed to inversely quantize the inputted matrix of original quantization coefficients level with the first quantization parameter Q1 to generate a matrix of de-quantization coefficients, referred to as xe2x80x9cdequantxe2x80x9d, i.e., DCT coefficients, for each of macroblocks as follows:                     dequant        =                              {                                          2                xc3x97                level                            +                              sign                ⁡                                  (                  level                  )                                                      }                    xc3x97                                                    Q                1                            xc3x97              QM                        32                    ⁢                      xe2x80x83                    ⁢          or                                    equation        ⁢                  xe2x80x83                ⁢                  (          1          )                                        dequant        =                  level          xc3x97                                                    Q                1                            xc3x97              QM                        16                                              equation        ⁢                  xe2x80x83                ⁢                  (          2          )                    
where the equation (1) is used for the inter macroblock, while the equation (2) is used for the intra macroblock. QM is a matrix of quantization parameters stored in a predetermined quantization table. The first quantization parameter Q1 and the matrix of quantization parameters QM are derived from the inputted first bit streams b1 by the decoder 51. Here, the original quantization coefficients level, the de-quantization coefficients dequant, the matrix of quantization parameters QM, and the first quantization parameter Q1 are integers. The de-quantization coefficients dequant calculated by the equations (1) and (2) should be rounded down to the nearest one.
The quantizer 55 is electrically connected to the de-quantizer 53 and designed to input the matrix of de-quantization coefficients dequant from the de-quantizer 53 and then quantize the inputted matrix of de-quantization coefficients dequant for each of macroblocks with a second quantization parameter, referred to as xe2x80x9cQ2xe2x80x9d hereinlater, to generate a matrix of re-quantization coefficients, referred to as xe2x80x9ctlevelxe2x80x9d, as follows:                     tlevel        =                  dequant          xc3x97                      16                                          Q                2                            xc3x97              QM                                ⁢                      xe2x80x83                    ⁢          or                                    equation        ⁢                  xe2x80x83                ⁢                  (          3          )                                        tlevel        =                              dequant            xc3x97                          16                                                Q                  2                                xc3x97                QM                                              +                                    sign              ⁡                              (                dequant                )                                      xc3x97                          1              2                                                          equation        ⁢                  xe2x80x83                ⁢                  (          4          )                    
where the equation (3) is used for the inter macroblock, while the equation (4) is used for the intra macroblock. The second quantization parameter Q2 is obtained by the rate controller 59. Here, the re-quantization coefficients tlevel and the second quantization parameter Q2 are also integers. The re-quantization coefficients tlevel calculated by the equations (3) and (4) should be rounded down to the nearest one. Such rounding operation for the integers will be omitted from the later description for avoiding tedious repetition.
The variable length encoder 57 is electrically connected to the quantizer 55 and designed to input the re-quantization coefficients level from the quantizer 55 and then encode the inputted matrix of the re-quantization coefficients tlevel to generate an objective picture data for each of pictures to sequentially output the objective picture data in the form of the second bit streams b2 through the output terminal a2. The variable length encoder 57 is further electrically connected to the variable length decoder 51 and designed to input a diversity of information included in the first bit streams b1 necessary for the second bit streams b2 from the variable length decoder 51.
The rate controller 59 is electrically connected to the de-quantizer 53 and designed to perform a rate control over the encoding in the conventional transcoder 50 according to the TM-5 on the basis of the information obtained from the de-quantizer 53 as described below.
Referring to FIG. 14 of the drawings, there is shown a flowchart of the rate controlling process according to TM-5 in the conventional transcoder 50. As shown in FIG. 14, the rate controlling process comprises steps A1 to A14.
In the step A1, xe2x80x9c1xe2x80x9d is assigned to the picture number variable n representing a serial number of pictures within the first bit streams b1. Hereinlater, the n-th picture in the first bit streams b1 is referred to as xe2x80x9cpic(n)xe2x80x9d.
In the following step A2, a global complexity measure Xi, Xp, or Xb for the corresponding type of pictures, i.e., I, P or B-picture is computed as follows:
Xi=Sixc3x97Qixe2x80x83xe2x80x83equation (5)
or
Xp=Spxc3x97Qpxe2x80x83xe2x80x83equation (6)
or
Xb=Sbxc3x97Qbxe2x80x83xe2x80x83equation (7)
where Si, Sp, or Sb is the number of bits generated by encoding I, P or B-picture, and Qi, Qp, or Qb is the average quantization parameter computed by averaging the actual quantization values used during the quantization of the all macroblocks within I, P or B-picture. The average quantization parameters Qi, Qp, and Qb are normalized within a range of 1 to 31. The average quantization parameters Qi, Qp, and Qb respectively correspond to the first quantization parameters Q1 obtained from the variable length decoder 51.
The global complexity measure Xi, Xp, or Xb of the corresponding picture is inversely proportional to the compressing ratio of the moving picture, namely, a ratio of the volume of information in the second bit streams b2 to that in the first bit streams b1. Namely, as the volume of information on the first bit streams b1 becomes larger, the compressing ratio is decreased. Therefore, the global complexity measure Xi, Xp, or Xb of the corresponding picture becomes larger, as the compressing ratio is decreased. In contrast, the global complexity measure Xi, Xp, or Xb of the corresponding picture becomes smaller, as the compressing ratio is increased.
The initial value of global complexity measure Xi, Xp, or Xb of the corresponding picture is given as follows:
Xi=160xc3x97Target_Bitrate/115xe2x80x83xe2x80x83equation (8)
or
Xp=60xc3x97Target_Bitrate/115xe2x80x83xe2x80x83equation (9)
or
Xb=42xc3x97Target_Bitrate/115xe2x80x83xe2x80x83equation (10)
where Target_Bitrate is measured in bits/s and corresponds to the target bit rate of the first conventional transcoder 50.
In the following step A3, the target number of bits for the next picture in the current GROUP OF PICTURES Ti, Tp, or Tb is computed as:                               T          i                =                              R                          1              +                                                                    N                    p                                    ⁢                                      X                    p                                                                                        X                    i                                    ⁢                                      K                    p                                                              +                                                                    N                    b                                    ⁢                                      X                    b                                                                                        X                    i                                    ⁢                                      K                    b                                                                                ⁢                      xe2x80x83                    ⁢          or                                    equation        ⁢                  xe2x80x83                ⁢                  (          11          )                                                  T          p                =                              R                                          N                p                            +                                                                    N                    b                                    ⁢                                      K                    p                                    ⁢                                      X                    b                                                                                        K                    b                                    ⁢                                      X                    p                                                                                ⁢                      xe2x80x83                    ⁢          or                                    equation        ⁢                  xe2x80x83                ⁢                  (          12          )                                                  T          b                =                  R                                    N              b                        +                                                            N                  p                                ⁢                                  K                  b                                ⁢                                  X                  p                                                                              K                  p                                ⁢                                  X                  b                                                                                        equation        ⁢                  xe2x80x83                ⁢                  (          13          )                    
where Np and Nb are the remaining number of P-pictures and B-pictures, respectively, remaining the current GROUP OF PICTURES in the encoding order. Kp and Kb are universal constants dependent on the quantization matrices. When it is assumed that Kp=1.0 and Kb=1.4, the quality of the image can be always optimized.
In the following step A4, the judgment is made whether the picture number variable n is xe2x80x9c1xe2x80x9d or not, i.e., the current picture is the first picture pic(1) or not. When the answer is xe2x80x9cYESxe2x80x9d, i.e., the current picture is the first picture pic(1), the step A4 goes to the step A5. When, on the other hand, the answer is xe2x80x9cNOxe2x80x9d, i.e., the current picture is not the first picture, the step A4 goes to the step A6. In the step A5, the remaining number of bits assigned to the GROUP OF PICTURES R is initialized according to the following equation (14). This remaining number of bits assigned to the GROUP OF PICTURES R is computed before encoding the first pictures pic(1) within the GROUP OF PICTURES, i.e., an I-picture, as follows:
R=Target_Bitratexc3x97NPIC/picture_rate+Rxe2x80x83xe2x80x83equation (14)
where NPIC is the total number of pictures in each of the GROUP OF PICTURES, and picture_rate is expressed in the number of decoded pictures per second. At the start of the sequence R=0.
In the step A6, the above remaining number of bits assigned to the GROUP OF PICTURES R is updated before encoding the current picture pic(n) as follows:
R=Rxe2x88x92Sixe2x80x83xe2x80x83equation (15)
or
R=Rxe2x88x92Spxe2x80x83xe2x80x83equation (16)
or
R=Rxe2x88x92Sbxe2x80x83xe2x80x83equation (17)
where Si, Sp, or Sb is the number of bits generated in the previous picture pic(nxe2x88x921) of the corresponding type (I, P or B) just encoded.
The step A5 or A6 goes to the step A7 wherein xe2x80x9c1xe2x80x9d is assigned to the macroblock number variable j (j greater than =1) representing a serial number of macroblocks within one of the pictures. Hereinlater the j-th macroblock in the picture is referred to as xe2x80x9cMB(j)xe2x80x9d.
In the following step A8, the fullness of appropriate virtual buffer di(j), dp(j) or db(j) is computed depending on the picture type before encoding macroblock MB(j) as follows:                                           d            i                    ⁡                      (            j            )                          =                                            d              i                        ⁡                          (              0              )                                +                      B            ⁡                          (                              j                -                1                            )                                -                                                                      T                  i                                xc3x97                                  (                                      j                    -                    1                                    )                                            NMB                        ⁢                          xe2x80x83                        ⁢            or                                              equation        ⁢                  xe2x80x83                ⁢                  (          18          )                                                              d            p                    ⁡                      (            j            )                          =                                            d              p                        ⁡                          (              0              )                                +                      B            ⁡                          (                              j                -                1                            )                                -                                                                      T                  p                                xc3x97                                  (                                      j                    -                    1                                    )                                            NMB                        ⁢                          xe2x80x83                        ⁢            or                                              equation        ⁢                  xe2x80x83                ⁢                  (          19          )                                                              d            b                    ⁡                      (            j            )                          =                                            d              b                        ⁡                          (              0              )                                +                      B            ⁡                          (                              j                -                1                            )                                -                                                    T                b                            xc3x97                              (                                  j                  -                  1                                )                                      NMB                                              equation        ⁢                  xe2x80x83                ⁢                  (          20          )                    
where B(jxe2x88x921) is the number of bits generated by encoding all macroblocks in the picture up to including (jxe2x88x921)th macroblock MB(jxe2x88x921). NMB is the total number of macroblocks in the picture. di(j), dp(j), or db(j) is a fullness of virtual buffer at the j-th macroblock MB(j) for I, P, or B-picture.
di(0), dp(0), or db(0) is an initial fullness of virtual buffer for I, P, or B-picture and given by:
di(0)=10xc3x97r/31xe2x80x83xe2x80x83equation (21)
or
dp(0)=Kpxc3x97di(0)xe2x80x83xe2x80x83equation (22)
or
db(0)=Kbxc3x97di(0)xe2x80x83xe2x80x83equation (23)
where r is referred to as xe2x80x9creaction parameterxe2x80x9d and, used for the control of a reaction rate of the loop of feed back as follows:
r=2xc3x97Target_Bitrate/picture_ratexe2x80x83xe2x80x83equation (24)
The final fullness of the virtual buffer di(NMB), dp(NMB), or db(NMB) of the NMB-th macroblock MB(NMB) of the current picture pic(n) is used as di(0), dp(0), or db(0) for encoding the first macroblock MB(1) within the next picture pic(n+1) of the same type.
In the following step A9, the reference quantization parameter Q(j) for the j-th macroblock MB(j) in each of pictures is computed on the basis of the above virtual buffer fullness d(j) as follows:
xe2x80x83Q(j)=d(j)xc3x9731/rxe2x80x83xe2x80x83equation (25)
Here, the reference quantization parameter Q(j) is identical with the aforesaid second quantization parameter Q2.
In the following step A10, the j-th macroblock MB(j) is quantized with the reference quantization parameter Q(j) computed in the step A9. In the following step A11, the macroblock number variable j is incremented by 1. The step A11 goes to the step A12 wherein the judgment is made whether the macroblock number variable j is more than the total number of macroblocks NMB within the nth picture pic(n) or not. When the answer is xe2x80x9cNOxe2x80x9d, i.e., the macroblock number variable j is less than the total number of macroblocks NMB, the control is returned from the step A12 to the step A8. When, on the other hand, the answer is xe2x80x9cYESxe2x80x9d, i.e., the macroblock number variable j exceeds the total number of macroblocks NMB, the step A12 goes to the step A13.
The macroblock number variable j thus serves as a loop counter for repeating the steps A8 to A11 to encode all the macroblocks MB(j) in the present picture pic(n). The entire macroblocks, the first macroblock MB(1) up to the NMB-th macroblock MB(NMB), in the nth picture pic(n) can be thus encoded in turn.
In the step A13, the picture number variable n is incremented by 1. Then the step A13 goes to the step A14 wherein the judgment is made whether the picture number variable n is more than the total number of pictures NPIC or not. When the answer is xe2x80x9cNOxe2x80x9d, i.e., the picture number variable n is less than the total number of pictures NPIC, the control is returned from the step A14 to the step A2. When, on the other hand, the answer is xe2x80x9cYESxe2x80x9d, i.e., the picture number variable n exceeds the total number of pictures NPIC, this routine of the rate controlling process is terminated. The picture number variable n thus serves as a loop counter for repeating the steps A2 to A13 to process all the pictures pic(n) in the present GROUP OF PICTURE. The entire pictures, the first pictures pic(1) up to the NPIC-th picture pic(NPIC), in the present GROUP OF PICTURE can be therefore processed in turn.
The aforesaid conventional transcoder 50, however, has no information on the structure of GROUP OF PICTURES, such as a picture rate of I or P-pictures within each of the GROUP OF PICTURES, so that the transcoder 50 must estimate the structure of GROUP OF PICTURES within the inputted moving picture sequence to allocate bits for each type of pictures within the estimated structure of GROUP OF PICTURES.
Furthermore, the first conventional transcoder 50 is required to decode the first bits streams b1 almost all over the layers, such as the sequence layer, the GROUP OF PICTURES layer, the picture layer, the slice layer, and the macroblock layer in order to derive necessary data for transcoding from the first bits streams b1. This operation wastes time, thereby causing the delay in the transcoding process.
Referring to FIG. 15 of the drawings, there is shown an example of the improvement of the above transcoder 50 as a second conventional transcoder 60. The second conventional transcoder 60 is adapted to perform the rate control without estimating the structure of GROUP OF PICTURES. As shown in FIG. 15, the second conventional transcoder 60 comprises a delay circuit 61 and a rate controller 62 in addition to the variable length decoder 51, the de-quantizer 53, the quantizer 55 and the variable length encoder 57 same as those of the first conventional transcoder 50 shown in FIG. 13. These same constitutional elements are simply represented by the same reference numerals as those of the conventional transcoder 50, and will be thus omitted from description for avoiding tedious repetition.
The delay circuit 61 is interposed between the variable length decoder 51 and the de-quantizer 53 and designed to control the flow of the signal from the variable length decoder 51 to the de-quantizer 53. The delay circuit 61 is operated to delay starting the de-quantizing process in the de-quantizer 53 until the variable length decoder 51 has been finished to decode one of the pictures in the coded moving picture sequence signal.
As shown in FIG. 15, the rate controller 62 of the second conventional transcoder 60 includes a target ratio computing unit 63, an input bit summing unit 65, a bit difference computing unit 67, a target output bit updating unit 69, and a quantization parameter computing unit 71.
The target ratio computing unit 63 is electrically connected to the variable length decoder 51 and designed to input an input bit rate of the first bit streams b1, hereinlater referred to as xe2x80x9cInput_Bitratexe2x80x9d, from the variable length decoder 51, and input a target bit rate, hereinlater referred to as xe2x80x9cTarget_Bitratexe2x80x9d through a terminal a3. Alternatively, the target bit rate Target_Bitrate may be previously stored in an internal memory, or determined on the basis of internal switches. The target ratio computing unit 63 is designed to then compute a target ratio (ioRatio) of the target bit rate Target_Bitrate to the input bit rate Input_Bitrate for each of pictures as follows.                     ioRatio        =                  Target_Bitrate          Input_Bitrate                                    equation        ⁢                  xe2x80x83                ⁢                  (          26          )                    
The input bit summing unit 65 is designed to sum the number of inputting bits of the picture decoded by the variable length decoder 51 to produce the total number of inputting bits Tin. On the other hand, the target output bit updating unit 69 is designed to compute a target number of outputting bits Tout to be generated by the variable length encoder 57. The target number of outputting bits Tout is computed by multiplying the total number of inputting bits Tin by the target ratio ioRatio as follows:
Tout=Tinxc3x97ioRatioxe2x80x83xe2x80x83equation (27)
The bit difference computing unit 67 is electrically connected to the variable length encoder 57 and the target output bit updating unit 69, and designed to input a real number of outputting bits Treal encoded by the variable length encoder 57 and input the target number of outputting bits Tout. The bit difference computing unit 67 is designed to then compute a difference number of bits Tdiff between the target number of outputting bits Tout and the real number of outputting bits Treal as follows:
Tdiff=Trealxe2x88x92Toutxe2x80x83xe2x80x83equation (28)
The target output bit updating unit 69 is electrically connected to the target ratio computing unit 63, the input bit summing unit 65, and the bit difference computing unit 67. The target output bit updating unit 69 is designed to update the target number of outputting bits To on the basis of the difference number of bits Tdiff as follows:
Tout=Toutxe2x88x92Tdiffxe2x80x83xe2x80x83equation (29)
The quantization parameter computing unit 71 is electrically connected to the target output bit updating unit 69 and designed to compute the reference quantization parameter Q(j) for each of macroblocks MB(j) on the basis of the target outputting bits Tout updated by the target output bit updating unit 69 in accordance with the step II of the TM-5.
FIG. 16 shows the flowchart of the rate controlling process performed by the above conventional transcoder 60. The rate controlling process in the transcoder 60 comprises the steps B1 to B13. The steps B6 to B13 are the almost same as those of the steps A7 to A14, respectively, in the rate controlling process shown in FIG. 14 except for the step B7 wherein the virtual buffer fullness is computed on the basis of the target number of outputting bits Tout given by the target output bit updating unit 69 instead of the target number of bits Ti, Tp or Tb obtained in the step A3 shown in FIG. 14. These same steps will be thus omitted from description for avoiding tedious repetition.
In the step B1, xe2x80x9c1xe2x80x9d is assigned to the picture number variable n. The step B1 then goes to the step B2 wherein the target ratio ioRatio is computed by the above equation (26). In the following step B3, the difference number of bits Tdiff is computed for the present picture pic(n) by the above equation (28). The step B3 then goes to the step B4 wherein the number of inputting bits Tin are summed up within the first bit streams b1. In the step B5, the target number of outputting bits Tout is computed by the above equation (27), and further updated by the above equation (29).
In the second conventional transcoder 60 thus constructed, the de-quantizer 53, however, must wait until the decoding process of the picture has been completed over the entire target transcoding flame, thereby causing the delay in the transcoding process.
Referring to FIGS. 17 and 18 of the drawings, there is shown another example of improvement of the above transcoder 50 as a third conventional transcoder 80. The third conventional transcoder 80 is also adaptable to perform the rate control without estimating the structure of GROUP OF PICTURES. As shown in FIG.17, the third conventional transcoder 80 comprises an input terminal a1 electrically connected to a first transmitting path and designed to input an input bit streams b3 at the input bit rate, and an output terminal as electrically connected to a second transmitting path and designed to output an output bit streams b4 at the target bit rate. In the third conventional transcoder 80, the input bit streams b3 have a format, non-adaptable for the MPEG-2, different from that of the bit streams b1 of the first and second conventional transcoders 50 and 60. The input bit streams b3 have information on the number of coding bits previously recorded thereon by the encoder, not shown.
The third conventional transcoder 80 comprises a variable length decoder 81 electrically connected to the input terminal a1, and a rate controller 82 in addition to the de-quantizer 53, the quantizer 55, the variable length encoder 57 which are same as those of the second transcoder 60 shown in FIG. 13. The rate controller 82 includes a target output bit updating unit 83, and a quantization parameter computing unit 85 in addition to the target ratio computing unit 63, and the bit difference computing unit 67 which are same as those of the second transcoder 60 shown in FIG. 15.
The third conventional transcoder 80 thus constructed can perform the rate control on the basis of the formation on the number of coding bits previously recorded in the input bit streams b3. The variable length decoder 81 is adapted to decode the pictures in the coded moving picture sequence signal within the third bit streams b3 to obtain the information on the number of coding bits and transmit this information to the de-quantizer 53. The variable length decoder 81 is also adapted to transmit the number of inputting bits Tin to the target output bit updating unit 83.
The outputting bit updating unit 83 is designed to compute the target number of outputting bits Tout is computed on the basis of the number of inputting bits Tin and the target ratio ioRatio by the above equation (26). The quantization parameter computing unit 85 is designed to compute the reference quantization parameter Q(j) for each of the macroblocks MB(j) on the basis of the target number of outputting bits Tout updated by the outputting bit updating unit 83 according to the step II in the TM-5. The quantizer 55 is then operated to quantize the j-th macroblock MB(j) on the basis of the reference quantization parameter Q(j) given by the quantization parameter computing unit 85.
FIG. 18 shows the flowchart of the rate controlling process performed by the above third conventional transcoder 80. The rate controlling process in the transcoder 80 comprises the steps C1 to C13. All the steps C1 to C13 are the same as those of the steps B1 to B13, respectively, in the rate controlling process shown in FIG. 16 except for the step C4 wherein the information on the number of inputting bits Tin in the current picture pic(n) is derived from the third bit streams b3 by the decoder 81 to obtain the total number of inputting bits Tin.
The third conventional transcoder 80 thus constructed has information on the number of coding bits previously recorded in the third bits streams b3 thereby making it possible to solve the problem of the delay in the second conventional transcoder 60. The third conventional transcoder 80, however, has another problem to restrict the form of the inputted bit streams. Moreover, the encoder which is linked with the third transcoder 80 must provide with the above information on the number of coding bits to be recorded in the bit streams, thereby causing the delay of process in the encoder.
In the any conventional transcoders 50, 60 and 80, the matrix of the de-quantization coefficients dequant is necessary for only the quantizer 55, but unnecessary for the transcoder itself to generate the desired bit streams. In order to eliminate the redundant matrix of the de-quantization coefficients dequant, there is proposed a fourth conventional transcoder 90 comprising a level converter 91 instead of the de-quantizer 53 and the quantizer 55 of the transcoder 50, as shown in FIG. 19. The level converter 91 is interposed between the variable length decoder 51 and the variable length encoder 57. The level converter 91 is designed to input the original picture data for each of pictures. The original picture data includes a matrix of original quantization coefficients level for each of macroblocks within the corresponding picture. The level converter 91 is electrically connected to the rate controller 59 and designed to input the second quantization parameter Q2 from the rate controller 59.
The level converter 91 is further designed to convert the original picture data for each of pictures including the matrix of original quantization coefficients level into the objective picture data including the matrix of re-quantization coefficients tlevel without generating the matrix of the de-quantization coefficients dequant. The following equations (30) and (31) for the matrix of re-quantization coefficients tlevel are lead by eliminating the matrix of the de-quantization coefficients dequant from the above equations (1), (2), (3) and (4).                     tlevel        =                  {                                    (                              level                +                                                      sign                    ⁡                                          (                      level                      )                                                        xc3x97                                      1                    2                                                              }                        xc3x97                                          Q                1                                            Q                2                                      ⁢                          xe2x80x83                        ⁢            or                                              equation        ⁢                  xe2x80x83                ⁢                  (          30          )                                        tlevel        =                              level            xc3x97                                          Q                1                                            Q                2                                              +                                    sign              ⁡                              (                level                )                                      2                                              equation        ⁢                  xe2x80x83                ⁢                  (          31          )                    
where the above equation (30) is used for the inter macroblock, while the above equation (31) is used for the intra macroblock. The level converter 91 is thus operable to convert the original picture data, for each of pictures, into the second pictures data with the first quantization parameter Qi and the second quantization parameter Q2. The first quantization parameter Qi is decoded from the first bit streams b1 by the variable length decoder 51, while the second quantization parameter Q2 is obtained from the rate controller 59.
In the fourth conventional transcoder 90, the rate controller 59 is designed to perform the rate control over the encoding in the transcoder 90 according to the TM-5. The variable length encoder 57 is electrically connected to the level converter 91 and to input the above matrix of re-quantization coefficients tlevel from the level converter 91.
The fourth conventional transcoder 90 thus constructed can efficiently perform the transcoding process at high speed without storing the matrix of de-quantization coefficients dequant in a memory.
The above conventional transcoders 50, 60, 80 and 90, however, has another problem with the rate-distortion performance in converting the quantization level. The detailed description of this problem will be made later. In short, the rate-distortion performance in converting the quantization level is unstable and variable in accordance with the first and second quantization parameters and the level of the original quantization coefficients level. Therefore, as the reduced information volume becomes larger, the quantization error is liable to increase, thereby causing the unstable rate control in transcoding.
It is therefore an object of the present invention to provide a method of transcoding a coded moving picture sequence in which there is no need of information excluded from the coded moving picture sequence and no limitation in the form of the coded moving picture sequence prepared by the encoder in order to real timely and effectively reduce the bit rate of the coded moving picture sequence with low delay.
It is another object of the present invention to provide a method of transcoding a coded moving picture sequence in which it is possible to reduce the quantization error as well as to increase the reduced information volume. Therefore, the above method can achieve the appropriate rate control.
It is further object of the present invention to provide an apparatus for transcoding a coded moving picture sequence in which there is no need of information excluded from the coded moving picture sequence and no limitation in the form of the coded moving picture sequence prepared by the encoder in order to real timely and effectively reduce the bit rate of the coded moving picture sequence with low delay.
It is also object of the present invention to provide an apparatus for transcoding a coded moving picture sequence in which it is possible to reduce the quantization error as well as to increase the reduced information volume. Therefore, the above apparatus can achieve the appropriate rate control.
It is a yet object of the present invention to provide a computer program product for transcoding a coded moving picture sequence signal in which there is no need of information excluded from the coded moving picture sequence signal and no limitation in the form of the coded moving picture sequence signal prepared by the encoder in order to real timely and effectively reduce the bit rate of the coded moving picture sequence with low delay.
It is further yet object of the present invention to provide a computer program product for transcoding a coded moving picture sequence signal in which it is possible to reduce the quantization error as well as to increase the reduced information volume. Therefore, the above computer program product can achieve the appropriate rate control.
In accordance with a first aspect of the present invention, there is provided a method of transcoding a coded moving picture sequence, comprising the steps of:
(a) inputting a first coded moving picture sequence signal at an input bit rate, the first coded moving picture sequence signal consisting of a series of pictures;
(b) computing an estimated number of inputting bits to be input in the inputting step (a) for a predetermined duration on the basis of the input bit rate of the first coded moving picture sequence signal;
(c) decoding the first coded moving picture sequence signal inputted in the inputting step (a) to reconstruct an original picture data, for each of the pictures, having a number of real inputting bits;
(d) computing, for each of the pictures, a remaining number of inputting bits after the previous picture in the first coded moving picture sequence signal is decoded in the decoding step (c) on the basis of the number of real inputting bits of the previous picture and the estimated number of inputting bits obtained in the computing step (b);
(e) transforming, on the basis of a scaling factor, the original picture data reconstructed in the decoding step (c) for each of the pictures into an objective picture data having a number of real outputting bits less than the number of real inputting bits of the original picture data;
(f) encoding the objective picture data obtained in the transforming step (e) to generate a second coded moving picture sequence signal;
(g) outputting the second coded moving picture sequence signal generated in the encoding step (f) at a target bit rate lower than the input bit rate of the first coded moving picture sequence signal;
(h) computing an estimated number of outputting bits to be output in the outputting step (g) for the predetermined duration on the basis of the target bit rate of the second coded moving picture sequence signal;
(i) computing, for each of the pictures, a remaining number of outputting bits after the previous picture in the second coded moving picture sequence signal is encoded in the encoding step (f) on the basis of the number of real outputting bits of the previous picture and the estimated number of outputting bits obtained in the computing step (h);
(j) computing, for each of the pictures, a target ratio of the remaining number of outputting bits computed in the computing step (i) to the remaining number of inputting bits computed in the computing step (d) before the present picture is transformed in the transforming step (e); and
(k) computing the scaling factor on the basis of the target ratio computed in the target ratio computing step (j) to control the target bit rate of the second coded moving picture sequence signal.
In the aforesaid method, the computing step (d) may comprise the steps of:
(d1) initializing the remaining number of inputting bits of the current picture on the basis of the estimated number of inputting bits;
(d2) updating the remaining number of inputting bits of the current picture on the basis of the number of real inputting bits of the previous picture and the remaining number of inputting bits of the previous picture;
(d3) repeating the step (d2) for each of the pictures; and
(d4) repeating the steps (d1) and (d3) at time intervals of the predetermined duration. Furthermore, the computing step (h) may comprise the steps of:
(h1) initializing the remaining number of outputting bits of the current picture on the basis of the estimated number of outputting bits;
(h2) updating the remaining number of outputting bits of the current picture on the basis of the number of real outputting bits of the previous picture and the remaining number of outputting bits of the previous picture;
(h3) repeating the step (h2) for each of the pictures; and
(h4) repeating the steps (h1) and (h3) at time intervals of the predetermined duration.
Alternatively, the aforesaid method may further comprise the step of repeating the steps (b), (d), (h) and (i) for each of the pictures.
Alternatively, in the aforesaid method, each of the pictures may consist of a plurality of blocks. Furthermore, the decoding step (c) may have the step of deriving an original scaling factor from the first coded moving picture sequence signal. The transforming step (e) may comprise the steps of:
(e1) dividing the original picture data for each of the picture into a plurality of block data respectively corresponding to the blocks;
(e2) quantizing the block data for each of the blocks with the scaling factor, and
(e3) repeating the steps (e1) and (e2) for each of the blocks within the picture to generate the objective picture data. The step (k) may have the step of updating the scaling factor on the basis of the original scaling factor.
In accordance with a second aspect of the present invention, there is provided a method of transcoding a coded moving picture sequence, comprising the steps of:
(a) inputting a first coded moving picture sequence signal at an input bit rate, the first coded moving picture sequence signal consisting of a series of pictures;
(b) decoding the first coded moving picture sequence signal inputted in the inputting step (a) to reconstruct an original picture data, for each of the pictures, having a number of real inputting bits, and derive an original scaling factor;
(c) transforming, on the basis of a scaling factor, the original picture data reconstructed in the decoding step (b) for each of the pictures into an objective picture data having a number of real outputting bits less than the number of real inputting bits of the original picture data;
(d) encoding the objective picture data obtained in the transforming step (c) to generate a second coded moving picture sequence signal;
(e) outputting the second coded moving picture sequence signal generated in the encoding step (d) at a target bit rate lower than the input bit rate of the first coded moving picture sequence signal;
(f) computing a temporary scaling factor on the basis of the reconstructed original picture data to control the target bit rate of the second coded moving picture sequence signal; and
(g) correcting the temporary scaling factor computed in the computing step (f) on the basis of the original scaling factor to derive the scaling factor for the transforming step (c).
In accordance with a third aspect of the present invention, there is provided an apparatus for transcoding a coded moving picture sequence, interposed between a first transmitting path and a second transmitting path, comprising: inputting means for inputting a first coded moving picture sequence signal through the first transmitting path at an input bit rate, the first coded moving picture sequence signal consisting of a series of pictures; decoding means for decoding the first coded moving picture sequence signal inputted through the inputting means to reconstruct an original picture data, for each of the pictures, having a number of real inputting bits; transforming means for transforming, on the basis of a scaling factor, the original picture data reconstructed by the decoding means for each of the pictures into an objective picture data having a number of real outputting bits less than the number of real inputting bits of the original picture data; encoding means for encoding the objective picture data obtained by the transforming means to generate a second coded moving picture sequence signal; outputting means for outputting the second coded moving picture sequence signal generated by the encoding means through the second transmitting path at a target bit rate lower than the input bit rate of the first coded moving picture sequence signal; and rate controlling means for controlling the target bit rate of the second coded moving picture sequence signal. The rate controlling means has: (a) a remaining bit computing unit for computing: an estimated number of inputting bits to be input through the inputting means for a predetermined duration on the basis of the input bit rate of the first coded moving picture sequence signal; a remaining number of inputting bits for each of the pictures in the first coded moving picture sequence signal on the basis of the number of real inputting bits of the current picture and the estimated number of inputting bits; an estimated number of outputting bits to be output through the outputting means for the predetermined duration on the basis of the target bit rate of the second coded moving picture sequence signal; and a remaining number of outputting bits for each of the pictures in the second coded moving picture sequence signal on the basis of the number of real outputting bits of the current picture and the estimated number of outputting bits; (b) a target ratio computing unit for computing, for each of the pictures, a target ratio of the remaining number of outputting bits computed by the remaining bit computing unit to the remaining number of inputting bits of the picture computed by the remaining bit computing unit; and (c) a scaling factor computing unit for computing the scaling factor on the basis of the target ratio computed by the target ratio computing unit to control the target bit rate of the second coded moving picture sequence signal.
The aforesaid apparatus may further comprise controlling means for operating the remaining bit computing unit to: update, for each of the pictures, the remaining number of inputting bits of the current picture on the basis of the number of real inputting bits of the previous picture and the remaining number of inputting bits of the previous picture; update, for each of the pictures, the remaining number of outputting bits of the current picture on the basis of the number of real outputting bits of the previous picture and the remaining number of outputting bits of the previous picture; initialize the remaining number of inputting bits of the current picture on the basis of the estimated number of inputting bits at time intervals of the predetermined duration; and initialize the remaining number of outputting bits of the current picture on the basis of the estimated number of outputting bits at time intervals of the predetermined duration.
Alternatively, the aforesaid apparatus may further comprise controlling means for operating the remaining bit computing unit to compute: for each of the pictures, the estimated number of inputting bits; the remaining number of inputting bits of the current picture; the estimated number of outputting bits; and the remaining number of outputting bits of the current picture.
Alternatively, in the aforesaid apparatus, each of the pictures may consist of a plurality of blocks. The decoding means may be further operated to derive an original scaling factor from the first coded moving picture sequence signal. The transforming means may be further operated to transform the original picture data for each of the picture into a block data for each of the blocks. The controlling means may be operated to update the scaling factor on the basis of the original scaling factor.
In accordance with a fourth aspect of the present invention, there is provided an apparatus for transcoding a coded moving picture sequence signal. The apparatus is interposed between a first transmitting path and a second transmitting path and comprises: inputting means for inputting a first coded moving picture sequence signal through the first transmitting path at an input bit rate, the first coded moving picture sequence signal consisting of a series of pictures; decoding means for decoding the first coded moving picture sequence signal inputted through the inputting means to reconstruct an original picture data, for each of the pictures, having a number of real inputting bits, and derive an original scaling factor, transforming means for transforming, on the basis of a scaling factor, the original picture data reconstructed by the decoding means for each of the pictures into an objective picture data having a number of real outputting bits less than the number of real inputting bits of the original picture data; encoding means for encoding the objective picture data obtained by the transforming means to generate a second coded moving picture sequence signal; outputting means for outputting the second coded moving picture sequence signal generated by the encoding means through the second transmitting path at a target bit rate lower than the input bit rate of the first coded moving picture sequence signal; rate controlling means for controlling the target bit rate of the second coded moving picture sequence signal by computing a temporary scaling factor on the basis of the reconstructed original picture data; and correcting means for correcting the temporary scaling factor computed by the rate controlling means on the basis of the original scaling factor to derive the scaling factor for transforming means.
In accordance with a fifth aspect of the present invention, there is provided a computer program product comprising a computer usable storage medium having computer readable code embodied therein for transcoding a coded moving picture sequence signal. The computer readable code comprises: (a) computer readable program code for inputting a first coded moving picture sequence signal at an input bit rate, the first coded moving picture sequence signal consisting of a series of pictures; (b) computer readable program code for computing an estimated number of inputting bits to be input by the computer readable program code (a) for a predetermined duration on the basis of the input bit rate of the first coded moving picture sequence signal; (c) computer readable program code for decoding the first coded moving picture sequence signal inputted by the computer readable program code (a) to reconstruct an original picture data, for each of the pictures, having a number of real inputting bits; (d) computer readable program code for computing, for each of the pictures, a remaining number of inputting bits after the previous picture in the first coded moving picture sequence signal is decoded by the computer readable program code (c) on the basis of the number of real inputting bits of the previous picture and the estimated number of inputting bits obtained by the computer readable program code (b); (e) computer readable program code for transforming, on the basis of a scaling factor, the original picture data reconstructed by the computer readable program code (c) for each of the pictures into an objective picture data having a number of real outputting bits less than the number of real inputting bits of the original picture data; (f) computer readable program code for encoding the objective picture data obtained by the computer readable program code (e) to generate a second coded moving picture sequence signal; (g) computer readable program code for outputting the second coded moving picture sequence signal generated by the computer readable program code (f) at a target bit rate lower than the input bit rate of the first coded moving picture sequence signal; (b) computer readable program code for computing an estimated number of outputting bits to be output by the computer readable program code (g) for the predetermined duration on the basis of the target bit rate of the second coded moving picture sequence signal; (i) computer readable program code for computing, for each of the pictures, a remaining number of outputting bits after the previous picture in the second coded moving picture sequence signal is encoded by the computer readable program code (f) on the basis of the number of real outputting bits of the previous picture and the estimated number of outputting bits obtained by the computer readable program code (h); (j) computer readable program code for computing, for each of the pictures, a target ratio of the remaining number of outputting bits computed by the computer readable program code (i) to the remaining number of inputting bits computed by the computer readable program code (d) before the present picture is transformed by the computer readable program code (e); and (k) computer readable program code for computing the scaling factor on the basis of the target ratio computed by the computer readable program code (j) to control the target bit rate of the second coded moving picture sequence signal.
In the aforesaid computer program product, the computer readable program code (d) may have: (d1) computer readable program code for initializing the remaining number of inputting bits of the current picture on the basis of the estimated number of inputting bits; (d2) computer readable program code for updating the remaining number of inputting bits of the current picture on the basis of the number of real inputting bits of the previous picture and the remaining number of inputting bits of the previous picture; (d3) computer readable program code for repeating the computer readable program code (d2) for each of the pictures; and (d4) computer readable program code for repeating the computer readable program codes (d1) and (d3) at time intervals of the predetermined duration.
The computer readable program code (j) may further comprise: (h1) computer readable program code for initializing the remaining number of outputting bits of the current picture on the basis of the estimated number of outputting bits; (h2) computer readable program code for updating the remaining number of outputting bits of the current picture on the basis of the number of real outputting bits of the previous picture and the remaining number of outputting bits of the previous picture; (h3) computer readable program code for repeating the computer readable program code (h2) for each of the pictures; and. (h4) computer readable program code for repeating the computer readable program codes (h1) and (h3) at time intervals of the predetermined duration.
Alternatively, the aforesaid computer program product may further comprise computer readable program code for repeating the computer readable program codes (b), (d), (h) and (i) for each of the pictures.
Alternatively, in the aforesaid computer program product, each of the pictures may consist of a plurality of blocks. The computer readable program code (c) may further have computer readable program code for deriving an original scaling factor from the first coded moving picture sequence signal. The computer readable program code (e) may comprise: (e1) computer readable program code for dividing the original picture data for each of the picture into a plurality of block data respectively corresponding to the blocks; (e2) computer readable program code for quantizing the block data for each of the blocks with the scaling factor; and (e3) computer readable program code for repeating the computer readable program codes (e1) and (e2) for each of the blocks within the picture to generate the objective picture data. The computer readable program code (k) has further computer readable program code for updating the scaling factor on the basis of the original scaling factor.
In accordance with a sixth aspect of the present invention, there is provided a computer program product comprising a computer usable storage medium having computer readable code embodied therein for transcoding a coded moving picture sequence signal. The computer readable code comprises: (a) computer readable program code for inputting a first coded moving picture sequence signal at an input bit rate, the first coded moving picture sequence signal consisting of a series of pictures; (b) computer readable program code for decoding the first coded moving picture sequence signal inputted by the computer readable program code (a) to reconstruct an original picture data, for each of the pictures, having a number of real inputting bits, and derive an original scaling factor, (c) computer readable program code for transforming, on the basis of a scaling factor, the original picture data reconstructed by the computer readable program code (b) for each of the pictures into an objective picture data having a number of real outputting bits less than the number of real inputting bits of the original picture data; (d) computer readable program code for encoding the objective picture data obtained by the computer readable program code (c) to generate a second coded moving picture sequence signal; (e) computer readable program code for outputting the second coded moving picture sequence signal generated by the computer readable program code (d) at a target bit rate lower than the input bit rate of the first coded moving picture sequence signal; (f) computer readable program code for computing a temporary scaling factor on the basis of the reconstructed original picture data to control the target bit rate of the second coded moving picture sequence signal; and (g) computer readable program code for correcting the temporary scaling factor computed by the computer readable program code (f) on the basis of the original scaling factor to derive the scaling factor for the computer readable program code (c).